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(54) Apparatus and method for coding and decoding video images 



(57) An apparatus and method lor coding and 
decoding digital video images, capable of maintaining 
the quality of static regions, such as background 
images, in decoded pictures even when the resolution 
of pictures has been changed from high to low. The 
apparatus is equipped with two storage units, low-reso- 
lution and high-resolution picture storage units (4,3). to 
hold reference pictures in two different picture formats. 
When processing coded blocks (or blocks having at 
least one non-zero transform coefficient), a high-resolu- 
tion picture updating unit (13)converts corresponding 
blocks of a low-resolution reference picture retrieved 



from the low-resolution picture storage unit (4) to obtain 
high-resolution block images. H then updates a high- 
resolution picture stored in the high-resolution picture 
storage unit (3) with the obtained high-resolution block 
images. This updating operation is not applied on non- 
coded blocks. As a result only active regions corre- 
sponding to the coded blocks are updated within the 
picture stored in the high-resolution picture storage unit 
(3). while the remaining part, which may possibly be 
static background images, is preserved without losing 
their high visual quality. 
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Description 

The present invention generally relates to video 
coding apparatus, video decoding apparatus, video 
coding methods, and video decoding methods. More s 
particularly, the present invention relates to a video cod- 
ing apparatus that performs predictive coding of digital 
video signals, a video decoding apparatus that repro- 
duces original motion images from a predictive-coded 
video bitstream produced by the video coding appara- io 
tus, and coding and decoding methods implemented in 
those video coding and decoding apparatus. 

In the field of digital motion picture coding tech- 
niques employing hybrid coding algorithms, H.261, 
MPEG-1 and MPEG-2, for example, are well-acknowl- is 
edged international standards. H.261 has been formu- 
lated by the International Telecommunication Union 
Telecommunication Standardization Sector (ITU-T), 
while MPEG-1 and MPEG-2 have been defined by a 
committee named Moving Picture Experts Group under 20 
the auspices of the International Standards Organiza- 
tion (ISO). Hybrid coding algorithms are actually a com- 
bination of different coding techniques, which compress 
video signals by reducing their spatial and temporal 
redundancies in the following sequence: (1) a source 25 
picture is divided into blocks of picture elements (pels or 
pixels). (2) motion compensation and orthogonal trans- 
formation (e.g.. discrete cosine transform) are applied 
independently on each block of pels, and (3) transform 
coefficients are quantized and then compressed with 30 
entropy coding techniques. 

When a rapid motion or a full scene transition has 
happened during a sequence of video frames, the 
above-described hybrid video coder may suffer from an 
overwhelming amount of coded frame data exceeding a 35 
certain standard level allowed for each frame. In such a 
situation, the video coder will forcibly reduce the amount 
of coded data in an attempt to regulate it at the standard 
level def ined by the bandwidth of communication chan- 
nels used. This attempt, however, can lead to extreme 40 
degradation in video image quality and frequent frame 
drops, thus resulting in unacceptably poor pictures 
when reconstructed at the receiving ends. 

To solve the above problems, a video coding sys- 
tem is disclosed in the Japanese Patent Application 4$ 
Laid-open Publication No. 7-96566 (1995), for instance. 
In this proposed system, the video coder reduces the 
resolution of input frame signals to regulate the amount 
of coded frame data when a full scene change or a rapid 
motion has been observed in the middle of a sequence so 
of video frames. More specifically, the video coder 
divides a picture into blocks and estimates motion vec- 
tors of the individual blocks as parameters for motion- 
compensated frame prediction. It then calculates the 
average magnitude of the estimated motion vectors ss 
over the entire picture. Based on the average magnitude 
of motions, the proposed video coder determines at 
what picture resolution the present frame should be 
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encoded. This decision affects the format of both source 
and reference pictures; they have to be converted to the 
determined resolution before being subjected to the 
coding process. 

In addition to the above, another Japanese Patent 
Application Laid-open Publication No. 63-155896 
(1988) proposes a similar video coding system which 
encodes video signals after converting both source and 
reference pictures to a lower resolution. 

The reference picture serves as the basis for pre- 
dictive coding/decoding of future frames, ft should be 
noted, however, that the above-described conventional 
video coding systems subsample the entire area of this 
reference picture, when the picture resolution mode has 
changed from high resolution mode to low resolution 
mode. While efficiently reducing the coded data size, 
this picture format conversion causes an adverse side 
effect. In a videoconference, for example, most back- 
ground portions of video images do not change with 
time, and such static portions never contribute to tem- 
poral increase in the amount of video information. 
Therefore, the reduction of picture resolution is unnec- 
essary as far as the background portions are con- 
cerned. The conventional video coding devices, 
however, destroy the clarity of static background images 
by reducing the picture resolution, thus causing an 
intense degradation in visual quality. 

Taking the above into consideration, it would be 
desirable to provide an apparatus and a method for cod- 
ing and/or decoding digital video images, capable of 
maintaining the quality of static regions such as back- 
ground images, even when the resolution of coded pic- 
tures has changed from high to low. 

According to one aspect of the present invention, 
there is provided a video coding apparatus for perform- 
ing a predictive coding of digital video input signals in 
conjunction with an internal picture format conversion 
according to a picture resolution mode that is deter- 
mined by a resolution selection controller disposed as 
an integral part of the video coding apparatus. Here, the 
picture resolution mode can be either a high resolution 
mode or a low resolution mode. 

This proposed video coding apparatus comprises 
the following key elements: 

(a) a high-resolution picture storage unit to store a 
high-resolution picture that has been locally recon- 
structed in a high resolution format; 

(b) a low-resolution picture storage unit to store a 
low-resolution picture that has been locally recon- 
structed in a low resolution format; 

(c) a selective reading-out unit to selectively read 
out the high-resolution picture from the high-resolu- 
tion picture storage unit when the high resolution 
mode has been selected by the resolution selection 
controller, or the low-resolution picture from the 
low-resolution picture storage unit when the low 
resolution mode has been selected by the resolu- 
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tion selection controller; 

(d) a high-resolution picture updating unit to convert 
the low-resolution picture retrieved from the low- 
resolution picture storage unit into a high-resolution 
image when the low resolution mode is effective, 
and store the resultant high-resolution image into 
the high-resolution picture storage unit; and 

(e) a low-resolution picture updating unit to convert 
the high-resolution picture retrieved from the high- 
resolution picture storage unit into a low -resolution 
picture when the resolution selection controller has 
changed the picture resolution mode from the high 
resolution mode to the low resolution mode, and 
store the resultant low-resolution picture into the 
low-resolution picture storage unit. 

Another aspect of the present invention provides a 
video decoding apparatus for receiving and decoding a 
predictive-coded video bitstream produced by com- 
pressing digital video input signals with a predictive cod- 
ing technique in conjunction with an internal picture 
format conversion according to a picture resolution 
mode determined by a resolution selection controller. 

This video decoding .apparatus compri^J^e_fol-_ 
lowing key elements: 

(a) a high-resolution picture storage unit for storing 
a high-resolution picture that has been recon- 
structed in a high resolution format; 

(b) a low-resolution picture storage unit for storing a 
low-resolution picture that has been reconstructed 
in a low resolution format 

(c) a selective reading-out unit responsive to the 
picture resolution mode that has been decoded, for 
selectively reading out the high-resolution picture 
from the high-resolution picture storage unit when a 
high resolution mode is effective, or reading out the 
low-resolution picture from the low-resolution pic- 
ture storage unit when a low resolution mode is 
effective; 

(d) a high-resolution picture updating unit for, when 
the decoded picture resolution mode indicates that 
the low resolution mode is effective, converting the 
low-resolution picture retrieved from the low-resolu- 
tion picture storage unit to obtain high-resolution 
block images corresponding to coded blocks, and 
storing the high- resolution block images into the 
high-resolution picture storage unit; and 

(e) a low-resolution picture updating unit for con- 
verting the high-resolution picture retrieved from 
the high-resolution picture storage unit to obtain a 
low-resolution picture when the decoded picture 
resolution mode has changed from the high resolu- 
tion mode to the low resolution mode, and storing 
the obtained low-resolution picture into the low-res- 
olution picture storage unit. 

Reference is made, by way of example, to the 



accompanying drawings in which; 

FIG. 1 is a first conceptual view of an embodiment 
of the invention; 

FIG. 2 is a second conceptual view of an embodi- 
ment of the invention; 

FIG. 3 is a block diagram which shows a specific 
configuration of a first embodiment of the present 
invention; 

FIG. A is a diagram which explains a 2:1 downsam- 
pling process; 

FIG. 5 is a diagram which shows how a high-resolu- 
tion picture updating unit processes active coding 
blocks; 

FIG. 6 is a diagram which shows how the high-res- 
olution picture updating unit processes static cod- 
ing blocks; 

FIG. 7(A) is a diagram which illustrates a frame 
image saved in a QCIF picture storage unit 
FIG. 7(B) is a diagram which illustrates a frame 
image saved in a CIF picture storage unit; 
FIG. 8 is a diagram which explains 112 upsaro- 
pling process; 

FIG. 9 is a block diagram which shows a configura 



tion of a second embodiment of the present inven- 
tion; 

FIG. 10 is a block diagram which shows a specific 
configuration of a third embodiment of the present 
invention; and 

so FIG. 1 1 is a diagram which shows the internal struc- 
ture of a high-resolution picture updating unit. 

Several embodiments of the present invention will 
be described below with reference to the accompanying 
35 drawings. 

Referring first to FIG. 1. the concept of a first 
embodiment of the present invention will be explained 
below. This first embodiment relates to a video coder, 
which performs a motion-compensated predictive cod- 
40 ing of digital video input signals in conjunction with an 
internal picture format conversion according to a picture 
resolution mode that is determined by a resolution 
selection controller 1. Here, the picture resolution mode 
can be either a high resolution mode or a low resolution 
45 mode. The video coder comprises the following key ele- 
ments: 

(a) a high-resolution picture storage unit 3 to store 
a high-resolution picture that has been locally 

so reconstructed in a high resolution format; 

(b) a low-resolution picture storage unit 4 to store a 
low-resolution picture that has been locally recon- 
structed in a low resolution format 

(c) a selective reading-out unit 5 to selectively read 
55 out the high-resolution picture from the high-resolu- 
tion picture storage unit 3 when the high resolution 
mode has been selected by the resolution selection 
controller 1, or the low-resolution picture from the 
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low-resolution picture storage unit 4 when the low 
resolution mode has been selected by the resolu- 
tion selection controller 1 ; 

(d) a high-resolution picture updating unit 13 to con- 
vert the low-resolution picture retrieved from the s 
low-resolution picture storage unit 4 into a high-res- 
olution image and then store the resultant high-res- 
olution image into the high-resolution picture 
storage unit 3; and 

(e) a low-resolution picture updating unit 14 to con- io 
vert the high-resolution picture retrieved from the 
high-resolution picture storage unit 3 into a low-res- 
olution picture when the resolution selection con- 
troller 1 has changed the picture resolution mode 
from the high resolution mode to the low resolution 75 
mode, and store the resultant low-resolution picture 
into the low-resolution picture storage unit 4. 

Further, the high-resolution picture updating unit 13 
comprises: ~ go 

(d1) a first updating unit to convert the low-resolu- 
tion picture retrieved from the low-resolution picture 
storage unit 4 to obtain high-resolution block 
images corresponding only to coded blocks with 25 
non-zero motion vectors and intra-coded (i.e.. intra- 
frame coded) blocks, while the low resolution mode 
is effective, and store the high-resolution block 
images to the high-resolution picture storage unit 3, 
and 30 
{62) a second updating unit to convert, while the 
low resolution mode is effective, a prediction error 
signal having been reproduced by a dequantization 
& inverse orthogonal transformation unit 11, to 
obtain a high-resolution prediction error signal cor- 35 
responding only to coded blocks with zero motion 
vectors, reconstruct high-resolution block images 
by adding the high-resolution prediction error signal 
to corresponding blocks of the high-resolution pic- 
ture retrieved from the high-resolution picture stor- 40 
age unit 3. and update the high-resolution picture 
stored in the high-resolution picture storage unit 3 
with the reconstructed high-resolution block 
images. 

45 

The video coder of FIG. 1 further comprises the fol- 
lowing elements: a source picture resolution converter 
2, a prediction parameter calculation unit 6, a prediction 
picture generator 7, a prediction error signal generator 
8, an orthogonal transformation & quantization unit 9, a so 
codeword assignment unit 10. a dequantization & 
inverse orthogonal transformation unit 11, and a 
decoded picture generator 12. These elements, as well 
as the resolution selection controller 1, can also be seen 
in conventional video coding systems. 55 

The video coder has two internal operating modes 
concerning picture resolutions, as mentioned earlier. 
The resolution selection controller 1 determines at 



which resolution (i.e.. high resolution or low resolution) 
each source picture will be coded. The source picture 
resolution converter 2 then converts the resolution of a 
given source picture according to the resolution deter- 
mined by the resolution selection controller 1 . Here, a 
picture is divided into blocks having prescribed dimen- 
sions tor the purpose of block motion estimation. The 
prediction parameter calculation unit 6 compares each 
block of the picture supplied from the source picture res- 
olution converter 2 with another picture provided by the 
selective reading-out unit 5. Based on this comparison, 
the prediction parameter calculation unit 6 determines 
which coding scheme, intraframe coding or interframe 
coding, may be appropriate to each block, and then out- 
puts a motion vector H the interframe coding scheme 
has been selected. The prediction picture generator 7 
produces a prediction picture in a block-by-block man- 
ner, based on the output of the prediction parameter cal- 
culation unit 6. More specifically, the prediction picture 
generator 7 outputs zeros as the pel values of a predic- 
tion picture when the intraframe coding scheme is 
selected, while it constructs a prediction picture by 
applying the motion vectors calculated by the prediction 
parameter calculation unit 6 to aleference picture sup- 
plied from the selective reading-out unit 5 when the 
interframe coding scheme is effective. 

The prediction error signal generator 8 calculates 
the difference between the source picture output from 
the source picture resolution converter 2 and the above- 
mentioned prediction picture to generate a prediction 
error signal for every predetermined block. The orthog- 
onal transformation & quantization unit 9 performs an 
orthogonal transformation and quantization of the pre- 
diction error signal generated by the prediction error sig- 
nal generator 8. The codeword assignment unit 10 
receives, at least, the quantized prediction error signal 
from the orthogonal transformation & quantization unit 
9, the picture resolution mode from the resolution selec- 
tion controller 1 , and the intra/inter coding scheme and 
motion vectors from the prediction parameter calcula- 
tion unit 6. Cut of a predefined codeword table, the 
codeword assignment unit 10 retrieves codewords rele- 
vant to each combination of the above received data, 
thereby generating a coded video bitstream. The 
dequantization & inverse orthogonal transformation unit 
11 dequantizes the quantized prediction error signal 
supplied from the orthogonal transformation & quantiza- 
tion unit 9 and then applies an inverse orthogonal trans- 
formation to the dequantized signal, thereby 
reproducing the prediction error signal. The decoded 
picture generator 12 reconstructs a picture by summing 
the prediction picture and the prediction error signal 
reproduced by the dequantization & inverse orthogonal 
transformation unit 11 for each predetermined block. 
This, reconstructed picture is referred to as a decoded 
picture, and H is sent to either one of two different desti- 
nations, depending on the picture resolution mode 
determined by the resolution selection controller 1 . That 
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is. the decoded picture is stored into the high-resolution 
• picture storage unit 3 in the high resolution mode, or into 
the low-resolution picture storage unit 4 in the low reso- 
lution mode. 

The following will describe more specifically the 
operation ot the video coder configured as above. Sup- 
pose here that the resolution selection controller 1 has 
selected the high resolution mode lor the present frame. 
In the high resolution mode, the source picture resolu- 
tion converter 2 converts the source picture to the high 
resolution, and all switches shown in FIG. 1 are all set to 
the positions indicated by the broken lines. 

In the high resolution mode, the selective reading- 
out unit 5 reads out the picture from the high-resolution 
picture storage unit 3. and sends it to the prediction 
parameter calculation unit 6 and prediction picture gen- 
erator 7. The prediction parameter calculation unit 6 
compares the high- resolution source picture supplied 
by the source picture resolution converter 2 with the 
high-resolution reference picture provided by the selec- 
tive reading-out unit 5. and on the basis of this compar- 
ison. H determines which coding scheme to adopt - 
intraframe or interlrame. H the interlrame coding 
scheme is chosen, the prediction parameter calculation 
unit 6 calculates motion vectors. Note that the above 
process is performed independently for individual 
blocks constituting the picture. 

The prediction picture generator 7 produces a pre- 
diction picture according to the decision made by the 
prediction parameter calculation unit 6. In the case of 
intraframe coding, it simply outputs zeros as the pel val- 
ues since the prediction picture must be blank. In the 
interlrame coding, the prediction picture generator 7 
constructs a prediction picture by applying the motion 
vectors calculated by the prediction parameter calcula- 
tion unit 6 to the high resolution reference picture 
retrieved from the high-resolution picture storage unit 3 
by the selective reading-out unit 5. Note again that this 
image prediction process is performed on an individual 
block basis. 

The prediction error signal generator 8 calculates 
the difference between the high-resolution source pic- 
ture supplied from the source picture resolution con- 
verter 2 and the prediction picture produced by the 
prediction picture generator 7 for each predetermined 
block, thereby yielding the displaced block difference, or 
prediction error signal. This prediction error signal is 
transformed and quantized by the orthogonal transfor- 
mation & quantization unft 9, entropy-coded by the 
codeword assignment unit 10, and then reproduced by 
the dequantization & inverse orthogonal transformation 
unit 11, just in the same way as in a conventional video 
coder. 

The decoded picture generator 12 produces a 
decoded picture by summing the prediction picture gen- 
erated by the prediction picture generator 7 and the pre- 
diction error signal reproduced by the dequantization & 
inverse orthogonal transformation unit 1 1 on an individ- 



ual block basis. The resultant decoded picture is then 
stored into the high-resolution picture storage unit 3. 

Next, suppose that the resolution selection control- 
ler 1 has changed the picture resolution mode to the lo$f 
5 resolution mode to encode the next frame. Upon tra na- 
tion in the picture resolution from high to loaf, the tow- 
resolution picture updating unit 14 entirely converts th© 
picture stored in the high-resolution picture storage unft 
3 to the low resolution, and feeds the resultant low-res- 
70 olution picture to the low-resolution picture storage unit 
4. 

Now that the low resolution mod© is effective, the 
source picture resolution converter 2 reduces the reso- 
lution of a source picture, and the switches shown In 
is FIG. 1 are all changed to the opposite positions irtd5- 
cated by the solid lines. Accordingly, the selective read- 
ing-out unit 5 reads the low-resolution picture out of th© 
low-resolution picture storage unit 4. and delivers it to 
the prediction parameter calculation unit 6 and predcc- 
20 tion picture generator 7. 

As described above, in this embc<firo®r*a. the tosf- 
resolution picture storage unit 4 acquires tho latest p5c- 
ture when the picture resolution mod® fe switched from 
high to low. The prediction parameter calculation unft 6 
25 receives this low-resolution picture Sho ns«f refer- 
ence picture through the selective reading-owl unft 5. 
and compares it with the low-resolution source pictur© 
supplied from the source picture resolution converter 2. 
The prediction parameter calculation unit S determirtss 
30 the intra/inter coding scheme and calculates motion 
vectors in the same way as in the high resolution mode. 
Here, the prediction picture generator 7. prediction error 
signal generator 8, orthogonal transformation a quanti- 
zation unit 9, the codeword assignment unit 10. and 
35 dequantization & inverse orthogonal transformation unit 
1 1 also play their respective roles as they do in the high 
resolution mode. 

The decoded picture generator 12 yields a picture 
by summing the prediction picture generated by the pre- 
40 diction picture generator 7 and the prediction error sig- 
nal reproduced by the dequantization & inverse 
orthogonal transformation unit 11 for each predeter- 
mined block. The resultant decoded picture is then 
stored into the low-resolution picture storage unit 4. 
45 Here, the high-resolution picture updating unit 13 con- 
verts the picture stored in the low-resolution picture 
storage unit 4 to the high resolution - not entirely, but 
only for the blocks that are being "coded 0 - and updates 
the picture stored in the high-resolution picture storage 
so unit 3 with the converted block image®. 

Here, it may be necessary to clarify what was 
meant by "coded" in the above. Recall that the coding 
process is applied independently on individual blocks 
that constitute a picture. After passing through the vari- 
55 ous process steps, blocks in the current picture are 
finally divided into two groups, coded blocks and non- 
coded blocks. The coded blocks denote such blocks 
that exhibit some variations when compared with the 
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previous picture (or the reference picture), and their 
irrterlrame difference components naturally cause some 
nonzero transform coefficients to be coded. With this 
definition, intra-coded blocks are also regarded as 
coded blocks When reconstructed, those coded blocks s 
will be used to update the previously used reference pic- 
ture. 

As opposed to the coded blocks, non-coded blocks 
exhibit no variations from the previous picture. Such 
blocks are never encoded, and thus the previous refer- io 
ence picture remains unchanged as far as the non- 
coded blocks are concerned. 

When coded blocks are in process, the high-resolu- 
tion picture updating unit 13 converts the resolution of 
corresponding blocks of the low-resolution picture is 
retrieved from the low-resolution picture storage unit 4 
to create high-resolution block images. It then updates 
the high-resolution picture in the high-resolution picture 
storage unit 3 with the created high-resolution block 
images. Since this picture updating operation is not per- 20 
formed as for the non-coded blocks, only active part cor- 
responding to the coded blocks will be updated within 
the high-resolution picture, while the remaining part, 
which may possibly be inactive background images, is 
preserved as is. 25 

The above-described functions of the high-resolu- 
tion picture updating unit 13 allows the latest reference 
picture to be maintained in the high-resolution picture 
storage unit 3. even while the video coder is running in 
the low resolution mode. The quality of this reference 30 
picture, however, may be partly degraded to some 
extent because the coded blocks are updated on the 
basis of the low-resolution coding cycles. Although 
some quality degradation is introduced to active blocks, 
the background part of the reference picture maintains 35 
its quality without being needlessly updated. 

This brings several advantages when the picture 
resolution mode changes from low resolution to high 
resolution. Upon mode transition, the picture stored in 
the high-resolution picture storage unit 3 begins to 40 
serve as the new reference picture for motion-compen- 
sated predictive coding. Because the quality of this ref- 
erence picture has been maintained up to this point, the 
mode transition will not cause any picture degradation 
in the background part, as opposed to conventional as 
video coders. Another advantage is the reduction of 
background image information to be transmitted. In con- 
ventional coders, a low-to-high resolution mode transi- 
tion requires the details (i.e.. high-frequency 
components) of background images to be coded and so 
transmitted. The video coder proposed in the present 
invention eliminates the need for sending such details, 
thus suppressing the increase in the coded video bit- 
stream to be transmitted over the communications 
channel. s$ 

To focus on the high-resolution picture updating unit 
13, it comprises two functional entities, a first updating 
unit and a second updating unit The first updating unit 



selectively processes coded blocks with non-zero 
motion vectors and intra-coded blocks as follows. When 
the resolution selection controller 1 has selected the low 
resolution mode, the first updating unit converts the res- 
olution of a picture retrieved from the low-resolution pic- 
ture storage unit 4 to obtain high-resolution block 
images. It then updates a picture stored in the high-res- 
olution picture storage unit 3 with the obtained high-res- 
olution block images. 

The second updating unit, on the other hand, deals 
with coded blocks with zero motion vectors as follows. 
When the resolution selection controller 1 has selected 
the low resolution mode, the second updating unit 
upsamples a prediction error signal reproduced by the 
dequantization & inverse orthogonal transformation unit 
11, thus obtaining high-resolution prediction error sig- 
nal. It reconstructs a picture by adding this high-resolu- 
tion prediction error signal to the high-resolution 
reference picture read out of the high-resolution picture 
storage unit 3. The second updating unit updates the 
picture in the high-resolution picture storage unit 3 with 
the resultant decoded picture. 

Theoretically, blocks with no motion, such as back- 
ground images, will all fall into the category of non- 
coded blocks. In the real world, however, there are 
some disturbances that affect the images of those back- 
ground blocks. They include: shadows of moving human 
bodies, changes in size of camera's iris opening, flickers 
of fluorescent lamps used for interior lighting, internal 
and external noises, etc. Those factors are likely to 
cause a fluctuation in luminance, resulting in some non- 
zero prediction errors that have to be coded. To main- 
tain the image quality of such background blocks as 
much as possible, it is proposed to classify the coded 
blocks into "static coded blocks" and "active coded 
blocks." The static coded blocks are particular coded 
blocks having no spatial motion but exhibiting some 
temporal variations in luminance as mentioned above, 
while the active coded blocks are the coded blocks as 
originally defined. 

The video coder of the present embodiment 
extracts active coded blocks from the present picture by 
testing each block as to whether it is a coded block hav- 
ing a non-zero motion vector, and whether H is subject 
to intraframe coding. The blocks that match either crite- 
rion are selectively supplied to the first updating unit in 
order to update the high-resolution reference picture as 
already explained. 

The video coder also extracts static coded blocte 
from the present picture by examining each block as to 
whether it is a coded block with a zero-valued motion 
vector, and supplies them to the second updating unit. 
The second updating unit, being activated in the low 
resolution mode, updates the static coded blocks in the 
high-resolution reference picture. That is. the second 
updating unit converts a prediction error signal repro- 
duced by the dequantization & inverse orthogonal trans- 
formation unit 11 to the high resolution, and 
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reconstructs block images by adding the converted pre- 
. diction error signal to corresponding blocks of the high- 
quality relerence picture read out ot the high-resolution 
picture storage unit 3. The reconstructed block images 
are used to update the contents in the high-resolution 
picture storage unit 3. This mechanism allows back- 
ground part ol the high-resolution picture to be updated 
without deterioration, properly tollowing its luminance 
variations. 

The following paragraphs provide more specific 
explanation about the first embodiment 

FIG. 3 is a block diagram which shows a typical 
video coder of the first embodiment The elements 
shown in FIG. 1 are associated with those in FIG. 3 as 
listed below: 

Resolution selection controller 1 (FIG. 1) 
* Resolution selection controller 31 (FIG. 3) 

• Source picture resolution converter 2 

« Downsampling unit 32a and Selection switch 32b 
High-resolution picture storage unit 3 
« CIF picture storage unit 33 
Low-resolution picture storage unit 4 
m QCIF picture storage unit 34 

• Selectiverreading^xitunit 5 
« Selection switch 35 

Prediction parameter calculation unit 6 
s Motion estimation unit 36 

• Prediction picture generator 7 

m Prediction picture generator 37a and Selection 
switch 37b 

Prediction error signal generator 8 
= Subtractor 38 

Orthogonal transformation & quantization unit 9 
= DCT processor 39a and Quantizer 39b 
Codeword assignment unit 1 0 
= Entropy coder 40 

Dequantization & inverse orthogonal transforma- 
tion unit 11 

= Dequantizer 41a and IDCt processor 41b 

Decoded picture generator 12 

= Adder 42 and Selection switch 45 

High-resolution picture updating unit 13 

= High-resolution picture updating unit 43 and 

Selection switch 45 

Low-resolution picture updating unit 14 

s Downsampling unit 44 and Selection switch 45 

The acronyms used in this list denote as follows: 

• CIF: Common Intermediate Format 

QCIF: Quarter Common Intermediate Format 

DCT: Discrete Cosine Transform 

IDCT: Inverse Discrete Cosine Transform 

In addition to the above-listed elements, the video 
coder of FIG. 3 includes a coding controller 46 and a 
coded data buffer 47. 



In operation, the resolution selection controller 31 
receives quantizer step size from the coding controller 
46. the amount of coded frame data from the entropy 
coder 40, and buffer occupancy status from the coded 

i data buffer 47. Based on the received information, it 
determines the best resolution at which the video coder 
can transmit the video signal without excessive frame 
drops or quality deterioration. In this example, source 
pictures are supplied in a high- resolution CIF video tor- 

10 mat (352 x 288 pels per frame for luminance compo- 
nents). Another format used inside the video coder is 
QCIF (176 x 144 pels per frame for luminance compo- 
nents). The resolution selection controller 31 selects the 
high resolution (CIF) mode as the default mode, and 

75 only when the coded transmission data has exceeded 
the predetermined standard amount, it choose6 the low 
resolution (QCIF) mode. There are some proposed 
decision-making processes for this resolution mode 
selection. The Japanese Patent Application No. 8- 

20 75605 (1 996). for example, describes a method applica- 
ble to the present invention. In FK3L 1. the selected res- 
olution (or resolution mode information) is distributed to 
the selection switches 32b. 35, and 45 and the entropy 
coder 40. 

25 The downsampling unit 32a applies a 2:1 down- 
sampling process to a high-resolution (CIF) source pic- 
ture so as to create a low-resolution (QCIF) picture. The 
details of this downsampling process are explained 
below with reference to FIG. 4. 

30 FIG. 4 depicts the 2:1 downsampling process, 
where white dots and lower-case alphabetic characters 
represent high-resolution CIF pels and their respective 
picture element (pel) values. Black dots and upper-case 
letters beside them represent low-resolution QCIF pels 

35 and their respective pel values. The downsampling 
process calculates the QCIF pel values A, B, C. and D 
by simply averaging the values of tour CIF pels sur- 
rounding each QCIF pel. For example, the pel value A is 
obtained by 

40 

A = (a+fc+e+0/4. 0) 

The downward conversion from high-resolution (CIF) to 
low-resolution (QCIF) is achieved through such a proc- 
45 ess. 

Referring back to FIG. 3. when the resolution selec- 
tion controller 31 has selected the high resolution (CIF) 
mode, the selection switch 32b is set to the position indi- 
cated by the broken line to select high-resolution (CIF) 

so source pictures, allowing source pictures to skip the 
downsampling process. On the other hand, when the 
resolution selection controller 31 has selected the low 
resolution (QCIF) mode, the selection switch 32b will be 
positioned as indicated by the solid line, thereby select- 

55 ing low-resolution (QCIF) output pictures generated by 
the downsampling unit 32a. 

The selection switch 35 operates according to the 
resolution mode indicated by the resolution selection 
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controller 31. In the high resolution (CIF) mode, the 
selection switch 35 is set to the position indicated by the 
broken line. In the low resolution (OCIF) mode, the 
selection switch 35 is set to the position indicated by the 
solid line. 5 

A series of video coding processes implemented in 
this video coder are based on the ITU-T recommenda- 
tion H.261 for videoconferencing. For instance, the 
motion estimation unit 36 processes a picture on an 
individual macroblock basis. Macroblocks are defined in 10 
the H.261 standards as structural units that constitute a 
frame image. In this patent specification, however, they 
are called "blocks" for simplicity. 

The motion estimation unit 36 calculates motion 
vectors as well as determining which to perform an is 
intraframe coding or interframe coding. The prediction 
picture generator 37a constructs a prediction picture by 
applying the motion vectors supplied from the motion 
estimation unit 36 to a reference picture provided 
through the selection switch 35. The selection switch so 
37b outputs different predicted images depending on 
the intra/inter coding scheme that has been determined 
for each individual block by the motion estimation unit 
36. That is. in the intraframe coding, it outputs zeros 
since neither motion compensation nor frame prediction 25 
takes effect in this coding scheme. On the other hand, 
the interframe coding makes the selection switch 37b 
select the prediction picture output of the prediction pic- 
ture generator 37a. 

The DCT processor 39a performs a discrete cosine 30 
transform (DCT) of a prediction error signal supplied 
thereto, thereby yielding a set of transform coefficients. 
The quantizer 39b quantizes the transform coefficients 
according to the quantizer step size sent from the cod- 
ing controller 46. The resultant values are referred to as 35 
quantized coefficients. 

The coding controller 46 receives information about 
the amount of the resultant coded data from the entropy 
coder 40, as well as being informed of buffer occupancy 
status from the coded data buffer 47. Using those two ao 
pieces of information, the coding controller 46 deter- 
mines quantizer step size and distributes it to the quan- 
tizer 39b, dequantizer 41a, resolution selection 
controller 31 , and entropy coder 40. 

The entropy coder 40 receives the quantized coeffi- 45 
cients from the quantizer 39b, the picture resolution 
mode from the resolution selection controller 31, the 
quantizer step size from the coding controller 46, and 
the irrtra/inter coding scheme and motion vectors from 
the motion estimation unit 36. Out of a predefined table, so 
the entropy coder 40 retrieves codewords relevant to 
the individual combinations of those received data, 
thereby outputting the coded frame data to the coded 
data buffer 47. This coded data buffer 47 serves as tem- 
porary storage for the coded frame data supplied from 55 
the entropy coder 40. 

The downsampling unit 44 has the same internal 
structure as that of the aforementioned downsampling 



unit 32a. The selection switch 45 is controlled in accord- 
ance with the picture resolution mode specified by the 
resolution selection controller 31. In the high resolution 
(CIF) mode, the two contacts in the selection switch 45 
are both set to the positions indicated by the broken 
lines. In the low resolution (QCIF) mode, they are 
switched to the opposite positions as indicated by the 
solid lines. As a transitional operation, the selection 
switch 45 keeps a connection path from the output of 
the downsampling unit 44 to the QCIF picture storage 
unit 34 just for a short time after the resolution mode 
has changed from high resolution (CIF) to low resolution 
(OCIF). This connection path allows the OCIF picture 
storage unit 34 to receive the latest low-resolution 
(QCIF) reference picture from the downsampling unit 44 
upon high-to-low transition of the resolution mode. 

Referring next to FIGS. 5 to 8, the following 
describes how the high-resolution picture updating unit 
43 and selection switch 45 operate. 

FIG. 5 shows how the high-resolution picture updat- 
ing unit 43 processes active coded blocks. Since the 
system is now attempting to reconstruct a picture in the 
low resolution (OCIF) mode, the two contacts in the 
selection switch 45 have been switched to the positions 
as indicated by the solid lines (FIG. 3). The motion esti- 
mation unit 36 provides the high-resolution picture 
updating unit 43 with information about a block to be 
processed, including the coding scheme selected and 
the motion vector calculated with respect to the block of 
interest. On the basis of such information, the high-res- 
olution picture updating unit 43 determines whether or 
not the block is an active coded block that meets either 
condition of: (a) a coded block with a non-zero motion 
vector, or (b) an intra-coded block. In the case that the 
block turned out to be an active coded block, the adder 
42 produces a low-resolution reconstructed image of 
the block by summing a low-resolution error signal 
received from the IDCT processor 41b and a low-reso- 
lution prediction picture obtained through the selection 
switch 37b. The block image reconstructed as such is 
then supplied to the QCIF picture storage unft 34 via the 
selection switch 45 that is set to the position indicated 
by the solid lines (FIG. 3). 

FIG. 7(A) illustrates a frame image saved in the 
QCIF picture storage unit 34. In FIG. 7(A), hatched 
blocks are coded blocks that exhibit some spatial 
motion. The non-coded blocks are static blocks such as 
background images. Due to the reduced resolution, any 
block images stored in the QCIF picture storage unit 34 
are of low quality. 

The high-resolution picture updating unit 43 con- 
tains an upsampling unit 43a, which applies a 12 
upsampling process to the low-resolution (OCIF) picture 
in the OCIF picture storage unft 34 to produce a high- 
resolution (CIF) picture. The following few paragraphs 
focus on this upsampling process, referring to FIG. 8. 

FIG. 8 shows a 1 2 upsampling process. In FIG. 8, 
black dots represent low-resolution (QCIF) pels, and 
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upper-case letters beside them indicate their respective 
pel values, while white dots represent high-resolution 
(CIF) pels, and lower-case alphabetic characters placed 
in the dots indicate their respective pel values. The hor- 
izontal and vertical dashed lines indicate boundaries 
between adjacent blocks. The upsampling process 
obtains the values of high-resolution (CIF) pels that are 
not immediately adjacent to the block boundaries by 
calculating a weighted average ot four QCIF pels sur- 
rounding each CIF pel ot interest. For example, the pel 
value f is obtained by 



f -(9A + 3B + 3C + D)/16. 



(2) 



where tour surrounding QCIF values A to D are 
summed up with appropriate weighting coefficients that 
have been determined in accordance with their respec- 
tive distances from the CIF pel of interest. 

In contrast to the above, when estimating the high- 
resolution (CIF) pels immediately adjacent to the block 
boundaries, the upsampling process never refers to any 
pel values in the adjacent blocks, but only uses two 
neighboring QCIF pels within the same block. For 
example, the pa* y5l ue ^js obtained by 



b = (3A + B)/4. 



(3) 



Take another pal value / for example. The upsam- 
pling unit 43a calculates it without referring to any pal 
values in the adjacent blocks but only considering two 
QCIF pal values A and C within the same block as 



g m {A +3C)/4. 



(4) 



Exceptionally, CIF pals at the corners of each block 
are estimated directly from the nearest QCIF pel as the 
following example. 

a* A (5) 



Referring back to FIG. 5, the high-resolution picture 
updating unit 43 now executes the above-described 
upsampling process for the active coded blocks as part 
of the low-resolution picture retrieved from the QCIF pic- 
ture storage unit 34. That is, the high-resolution picture 
updating unit 43 extracts active coded block from 
among the coded blocks shown in FIG. 7(A) and con- 
verts them into high-resolution (CIF) block images. 
Those high-resolution block images are then supplied to 
the CIF picture storage unit 33 via the selection switch 
45 that has been set to the positions indicated by the 
solid lines (FIG. 3). thereby updating the corresponding 
blocks in a selective fashion. 

FIG. 7(B) illustrates a frame image saved in the CIF 
picture storage unit 33. In FIG. 7(B). the hatched section 
indicates active blocks that have been updated by the 
high-resolution picture updating unit 43. The remaining 
non-updated blocks represent inactive regions such as 



background images. Thanks to such Hock updating 
opeiations. the CIF picture storage unit 33 always 
keeps the latest image even in the low resolution (QCIF) 
mode. Although the updated block images may have 
5 some quality degradation because they are based on 
low-resolution (QCIF) pictures, the quality ot non- 
updated blocks is maintained at a high level. 

FIG. 6 shows the operation of the high-resolution 
picture updating unit 43 when static coded blocks are 
w being processed. Since the system is now attempting to 
reconstruct a picture in the low resolution (QCIF) mode, 
the two contacts in the selection switch 45 are changed 
to the opposite positions as indicated by the solid lines 
(FIG. 3). The high-resolution picture updating unit 43 
is receives information about a block to be processed from 
the motion estimation unit 36. This block-specific infor- 
mation includes the coding scheme selected and the 
motion vector calculated with respect to the block of 
interest. On the basis of such information, the high-res- 
20 olution picture updating unit 43 determines whether or 
not the block of interest can be regarded as a static 
coded block. Here, static coded blocks have at least one 
non-zero DCT coefficient to be coded, although their 
respective motion vectors are zero vectors implying no 
25 " spatial displacement between frames. In the case that 
the block of interest is a static coded block, the adder 42 
produces a low-resolution image of that block by sum- 
ming a low-resolution reconstructed prediction error sig- 
nal received from the IDCT processor 41b and a low- 
so resolution prediction picture received through the selec- 
tion switch 37b. The block image reconstructed as such 
is then supplied to the QCIF picture storage unit 34 via 
the selection switch 45 that is set to the positions indi- 
cated by the solid lines (FIG. 3). 
ss As FIG. 6 shows, the high-resolution picture updat- 
ing unit 43 is equipped with another upsampling unit 
43b and an adder 43c, in addition to the aforementioned 
upsampling unit 43a. Having the same internal structure 
as that of the upsampling unit 43a, this second upsam- 
40 pling unit 43b applies a 1:2 upsampling process to the 
reconstructed prediction error signal to convert its reso- 
lution from low-resolution (QCIF) to high-resolution 
(CIF). 

When a static coded block is detected by the high- 
45 resolution picture updating unit 43, the upsampling unit 
43b upsamples the reconstructed prediction error signal 
received from the IDCT processor 41b. Subsequently, 
the adder 43c adds the upsampled reconstructed pre- 
diction error signal to a conesponding high-resolution 
so (CIF) block image retrieved from the CIF picture storage 
unit 33. thereby obtaining a high-resolution (CIF) recon- 
structed block image. This reconstructed block image is 
then supplied to the CIF picture storage unit 33 via the 
selection switch 45 that has been set to the positions 
55 indicated by the solid lines (FIG. 3). A corresponding 
block image stored in the CIF picture storage unit 33 is 
replaced with that reconstructed block image with the 
high resolution (CIF). Thanks to such block updating 
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operations, the CIF picture storage unit 33 always 
keeps the latest image even in the low resolution (OCIF) 
mode. Since the above -described block reconstruction 
scheme is based on a high-resolution (CIF) picture, the 
updated blocks can keep their original quality levels, 5 
while permitting temporal changes in luminance to be 
reflected therein. 

Incidentally, a video display unit may be coupled to 
the CIF picture storage unit 33 tor monitoring purposes. 
This video display unit, H available, will demonstrate 10 
improved video images whose background portions are 
maintained at high quality levels without being afleded 
by any changes in the picture resolution modes. 

As described above, the first embodiment of the 
present invention maintains the quality of inactive back- is 
ground images stored in the CIF picture storage unit 33 
even when the picture resolution is decreased to limit 
the amount of coded data within a predetermined stand- 
ard level. Therefore, the picture stored in the CIF picture 
storage unit 33. if monitored, will present clear back- 20 
ground images even if the internal picture resolution 
mode has changed from high to low. This feature brings 
an advantage to video decoders which reconstruct 
video images from coded-signals received from a video 
coder, which will be separately described later on as a 2s 
second embodiment of the present invention. 

Another advantage of the video coder proposed in 
the present invention is that it suppresses the increase 
in coded video data when the resolution mode has 
changed from low to high. In this situation, conventional 30 
video coders would produce a large amount of coded 
video information to regain the details of background 
images which have been lost during the past low-reso- 
lution coding operations. The present invention prevents 
this increase from happening, by maintaining a high- 35 
quality background image in the CIF picture storage unit 
33 even in the low resolution mode. 

Next, a second embodiment of the present inven- 
tion will be described below. The second embodiment of 
the present invention relates to a video decoder, whose 40 
configuration is shown in a block diagram of FIG. 9. 
Since the second embodiment is based on the concept 
and structure of the first embodiment, the following sec- 
tion will focus on its distinctive points, while affixing like 
reference numerals to like elements in FIG. 9. 45 

The video decoder in the second embodiment com- 
prises a local decoder 50 configured in the same way as 
the first embodiment, as well as having an entropy 
decoder 51 and a video display unit 52 as elements 
specific to video decoders. The entropy decoder 51 so 
reproduces quantized DCT coefficients, picture resolu- 
tion, quantizer step size, inter/tntra coding scheme, and 
motion vectors, by entropy-decoding a coded bitstream 
received from the sending end. The entropy decoder 51 
then distributes those reproduced signals to other func- 55 
tional blocks within the decoder system. More specifi- 
cally, it sends (a) the quantized DCT coefficients and 
quantizer step size to the dequantizer 41a. (b) the pic- 



ture resolution mode to the selection switches 35 and 
45, and (c) the inter/intra coding scheme and motion 
vectors to the prediction picture generator 37a. selec- 
tion switch 37b, and high-resolution picture updating 
unit 43. 

The video decoder of the second embodiment 
receives a coded bitstream produced at the video coder 
discussed in the first embodiment. The local decoder 50 
decodes the received bitstream, as does its counterpart 
in the video coder of the first embodiment. As a result, 
the CIF picture storage unit 33 always keeps video 
images with background regions maintained at high 
quality levels regardless of changes in the resolution 
modes. The video display unit 52 is configured to dis- 
play such video images. Therefore, the video encoder of 
the second embodiment maintains the quality of inac- 
tive background images, even when the picture resolu- 
tion is reduced so as to limit the amount of coded data 
within a predetermined standard level. The video dis- 
play unit 52 will keep a clear background image even if 
the picture resolution mode has changed from high to 
low. 

Next, a third embodiment of the present invention 
will be described below. Referring first to FIG 2, the fol- 
lowing section will present the concept of the thiid 
embodiment. The third embodiment relates to a video 
decoder, whose key elements include: 

(a) a high-resolution picture storage unit 22 to store 
a high-resolution picture that has been recon- 
structed in a high resolution format; 

(b) a decoded picture storage unit 23 to store a 
decoded picture that has been reconstructed; 

(c) a decoded picture generator 28 which produces 
a decoded picture based on a reproduced predic- 
tion error signal, stores the decoded picture into the 
decoded picture storage unit 23. and additionally 
stores the decoded picture into the high-resolution 
picture storage unit 22 only when the decoded pic- 
ture is of the high-resolution format; and 

(d) a high-resolution picture updating unit 29 which 
converts, when the decoded picture is of the low 
resolution format, only coded blocks of the decoded 
picture to obtain high-resolution block images cor- 
responding to the coded blocks, and stores the 
high-resolution block images into the high-resolu- 
tion picture storage unit 22. 

The high-resolution picture updating unit 29 com- 
prises a first updating unit and a second updating unit. 
The first updating unit selectively processes coded 
blocks with non-zero motion vectors and intra-coded 
blocks within a given decoded picture. When the low 
resolution mode is effective, this first updating unit con- 
verts the resolution of such blocks of the decoded pic- 
ture to obtain high-resolution block images and then 
stores them to the high-resolution picture storage unit 
22. 
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In contrast to the first updating unit the second 
updating unit deals with coded blocks with zero motion 
vectors. When the low resolution mode is set. the sec- 
ond updating unit converts a prediction error signal 
reproduced by the dequanti2ation & inverse orthogonal 
transformation unit 27 to yield a high-resolution predic- 
tion error signal, and reconstructs a picture by adding 
this high-resolution prediction error signal to the high- 
resolution picture retrieved from the high-resolution pic- 
ture storage unit 22. The second updating unit updates 
the picture in the high-resolution picture storage unit 22 
with the resultant reconstructed picture. 

The video decoder proposed in the third embodi- 
ment only introduces some additional circuits to a con- 
ventional video decoder. Actually, most elements 
illustrated in FIG. 2 are often seen in conventional 
decoder devices. Such conventional elements include 
the reconstruction unh 21 , decoded picture storage unit 
23. resolution converter 24. prediction picture generator 
26. dequarrtization & inverse orthogonal transformation 
unit 27. decoded picture generator 28. and video dis- 
play unh 30. In operation, the decoded picture storage 
unit 23 stores the decoded picture received from the 
decoded picture generator 28. no matter what resolu- 
tion h may have. The Vesblutibn converter 24 converts 
the reference picture stored in the decoded picture stor- 
age unit 23. according to the resolution mode indicated 
by the reconstruction unit 21. and sends the converted 
reference picture to the prediction picture generator 26. 

More specifically, when the high resolution mode is 
indicated by the reconstruction unit 21. the resolution 
converter 24 examines the resolution of the reference 
picture stored in the decoded picture storage unit 23. If 
it is a high-resolution picture, the resolution converter 24 
sends the reference picture to the prediction picture 
generator 26 without modification. If it is a low-resolu- 
tion picture, the resolution converter 24 converts it to the 
high resolution and sends the resultant high-resolution 
reference picture to the prediction picture generator 26. 
In turn, when the low resolution is indicated by the 
reconstruction unit 21, the resolution converter 24 
examines the resolution of the reference picture stored 
in the decoded picture storage unit 23. If it is a low-res- 
olution picture, the resolution converter 24 sends the 
reference picture to the prediction picture generator 26 
without modification. If "it is a high-resolution picture, the 
resolution converter 24 converts it to the low resolution 
and sends the resultant low-resolution picture to the 
prediction picture generator 26. 

The prediction picture generator 26 generates a 
prediction picture on an individual block basis, accord- 
ing to the intra/inter coding schemes reproduced by the 
reconstruction unit 21. More specifically, when a given 
block is an intra-coded block, the prediction picture gen- 
erator 7 outputs 2eros for the pel values of a prediction 
picture. In turn, when the block is a block coded by inter- 
frame prediction, the prediction picture generator 26 
constructs a prediction picture by applying a corre- 



sponding motion vector reproduced by the reconstruc- 
tion unit 21 to the reference picture supplied from the 
resolution converter 24. 

The prediction picture generated in either way is 
5 then supplied to the decoded picture generator 28. Th® 
decoded picture generator 28 calculates the sum of th© 
prediction picture received from the prediction pictur® 
generator 26 and the prediction error signal reproduced 
by the d equalization & inverse orthogonal transforms- 
io tion unit 27 on an individual block basis. The resultant 
decoded picture is then stored into the decoded pictur© 
storage unit 23. It should be added here that the repro- 
duced prediction error signal and the reference pictur® 
are consistent with the picture resolution mode repro- 
15 duced by the reconstruction unit 21. 

What have been described above are conventional 
decoding functions. The following paragraphs will 
describe the operation of the video coder, particularly 
about several elements that are newly introduced in th© 
so present invention. 

As mentioned earlier, it is desirable to provide view- 
ers with high-resolution video images tor at least inac- 
tive portions such as background images, even in th© 
situation that the system has to work in a to*/ resolution 
25 mode. On the other hand, there exists a compatibility 
issue that requires the video decoder to employ a con- 
ventional decoder loop consisting of the decoded p5c- 
ture storage unit 23. resolution converter 24. prediction 
picture generator 26. and decoded picture generator 28. 
30 This decoder loop has to work just in the same way as 
the local decoder loop works at the encoding end. To 
meet this requirement, the decoder loop is conf igured to 
operate as follows. 

When the resolution mode reproduced by the 
35 reconstruction unit 21 has changed from low to high, the 
resolution converter 24 converts the low-resolution pic- 
ture retrieved from the decoded picture storage unit 23 
into a high-resolution picture and sends it to the predic- 
tion picture generator 26. The prediction picture gener- 
40 ator 26 then produces a prediction picture. The decoded 
picture generator 28 constructs a decoded picture from 
the prediction picture and stores this high-resolution 
decoded picture into the decoded picture storage unit 
23. At the same time, the decoded picture is also sup- 
45 plied to the high- resolution picture storage unit 22 via a 
switch (28) illustrated in FIG. 2. After the transition to th© 
high resolution mode, high-resolution pictures are con- 
tinuously supplied from the decoded picture storage unit 
23 to the prediction picture generator 26, as long as th© 
so reconstruction unit 21 keeps indicating the same high 
resolution mode. 

When, in turn, the resolution mode has reversely 
changed from high to low. the resolution converter 24 
converts the high-resolution picture retrieved from the 
55 decoded picture storage unit 23 to a low-resolution pic- 
ture and sends it to the prediction picture generator 26. 
As a result, a low-resolution decoded picture is stored in 
the decoded picture storage unit 23. After this transition 



1U 



;OClD: <EP 08832©9A2J_> 



21 



EP 0 883 299 A2 



22 



1o the low resolution mode, low-resolution pictures are 
continuously supplied from the decoded picture storage 
unit 23 to the prediction picture generator 26. as long as 
the reconstruction unit 21 keeps indicating the same low 
resolution mode. 

In parallel with the above-described decoder loop, 
the high-resolution picture storage unit 22 and high-res- 
olution picture updating unit 29 play their unique roles in 
the following manner. In the low resolution mode, the 
high-resolution picture updating unit 29 selectively con- 
verts coded blocks in a decoded picture provided from 
the decoded picture generator 28 to raise their resolu- 
tion. With the resultant high-resolution block images, h 
then updates the picture stored in the high-resolution 
picture storage unit 22. As a result of this block-based 
updating operation, only an active part corresponding to 
the coded blocks is changed within the picture stored in 
the high-resolution picture storage unit 22, while the 
remaining part, which may possibly be a static back- 
ground image, is kept as is. without losing its high visual 
quality. The video decoder of FIG. 2 uses the high-reso- 
lution picture storage unit 22 as the signal source for the 
video display unit 30. This video display unit 30 will 
demonstrate improved video images with high-quality 
background portions without being affected by any 
changes in the resolution modes. 

To accomplish the above functions, the high-resolu- 
tion picture updating unit 29 comprises the first and sec- 
ond updating units. No further details of those two units 
are presented here again, because their functions are 
the same as those of the high-resolution picture updat- 
ing unit 13 (FIG. 1) in the first embodiment. 

Next, the following section will give a further expla- 
nation about the third embodiment of the present inven- 
tion by showing a more specific implementation. 

FIG. 10 is a block diagram which shows a specific 
configuration of the third embodiment The elements 
described in the conceptual view of FIG. 2 are associ- 
ated with the components shown in FIG. 10 as listed 
below: 

• Reconstruction unit 21 (FIG 2) 

■ Codeword decoder 61 (FIG 10) 
High-resolution picture storage unit 22 

« High-resolution picture storage unit 62 

Decoded picture storage unit 23 

= Frame memory 63 

Resolution converter 24 

= CIF/QCIF converter 64 

Prediction picture generator 26 

■ Prediction picture generator 66 
Dequantization & inverse orthogonal transforma- 
tion unit 27 

= Dequantizer 67a and IDCT processor 67b 

Decoded picture generator 28 

= Adder 68a and switch 68b 

High-resolution picture updating unit 29 

= High-resolution picture updating unit 69 and 
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Switch 71 
Video display unit 30 
* Video display unit 70 

5 The codeword decoder 61 reproduces quantized 

DCT coefficients, picture resolution mode, quantizer 
step size, interframe/intraframe coding scheme, and 
motion vectors, by decoding a coded video bitstream 
received from the sending end. It then delivers the 
io quantized DCT coefficients and quantizer step size to 
the dequantizer 67a, the picture resolution mode to the 
switches 68b and 71 and CIF/QCIF convener 64. and 
the interframe/intraframe coding scheme and motion 
vectors to the prediction picture generator 66 and high- 
is resolution picture updating unit 69. 

The CIF/QCIF converter 64 converts the resolution 
of a picture read out of the frame memory 63 to give the 
resolution specified by the codeword decoder 61 
regardless of its original resolution. More specifically, 
20 when the high resolution (CIF) mode is specified, the 
CIF/QCIF converter 64 upsamples the picture so that it 
will have the high resolution (CIF) as specified. When 
the low resojution^QCIF^ specified, the 

CIF/QCIF converter 64 downsamples the picture so that 

25 it will have the low resolution (QCIF). 

The switch 68b is closed only when the codeword 
decoder 61 has specified the high resolution (CIF) 
mode. Through this data path, a high-resolution (CIF) 
decoded picture is transferred block by block to the 

30 high-resolution picture storage unit 62, every time it is 
produced by the adder 68a. Therefore, the high-resolu- 
tion picture storage unit 62 holds the latest decoded pic- 
ture of high resolution (CIF) as long as the decoding 
process is performed in the high resolution (CIF) mode. 

35 In contrast to this, the low resolution (QCIF) mode 
requires the high-resolution picture storage unit 62 and 
related circuits to perform more complex operations. 
The next section will describe how the decoded picture 
is controlled in the low resolution mode. 

40 FIG. 1 1 shows the internal structure of the high-res- 
olution picture updating unit 69. The high-resolution pic- 
ture updating unit 69 comprises two upsampling units 
69a and 69b, an adder 69c. and a selection switch 69d. 
The upsampling units 69a and 69b execute upsampling 

45 processes in the same way as explained in the first 
embodiment (FIG. 8). The selection switch 69d is con- 
trolled in accordance with the intra/inter coding 
schemes and motion vectors received from the code- 
word decoder 61. More specifically, the high-resolution 

so picture updating unit 69 determines whether each given 
block is an active coded block or a static coded block, by 
checking the coding schemes and motion vectors. If the 
block of interest has turned out to be an active coded 
block, the selection switch 69d will be set to a position 

55 as indicated by the broken line in FIG. 1 1 . If the block is 
recognized as a state coded block, the selection switch 
69d will be set to the other position as indicated by the 
solid line in FIG. 11. In this context, the terms "active 
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coded block- and "static coded block- are used in the 
same meanings as defined in the first embodiment. 

A switch 71 is responsive to the resolution mode 
decoded by the codeword decoder 61, which allows the 
high-resolution picture updating unit 69 to send its out- 
put to the high-resolution picture storage unit 62 only in 
the low resolution (OCIF) mode. 

While the low resolution (OCIF) mode is effective, 
the circuit ol FIG. 11 operates as follows. When the 
block of interest is found to be an active coded block, the 
contact of the selection switch 69d is set to the position 
as indicated by the broken line in FIG. 11. Inside the 
high-resolution picture updating unit 69. the first upsam- 
pling unit 69a upsamples the low-resolution (QCIF) 
decoded picture sent from the adder 68a. to create a 
high-resolution (CIF) block image, and delivers it to the 
high-resolution picture storage unit 62 via the selection 
switch 69d. This high-resolution block image replaces 
the corresponding block of the picture stored in the 
high-resolution picture storage unit 62. 

The above-described updating process permits the 
high-resolution picture storage unit 62 to have the latest 
picture even when the decoder loop has been working 
in the low resolution (QCIF) mode. Although the 
updated blocks may be degraded to some extent 
because their source is low resolution (OCIF) pictures, 
the remaining non-updated blocks will keep their visual 
quality. 

Now imagine that the block of interest is found to be 
a static coded block. Then the contact of the selection 
switch 69d will be moved to the opposite side as indi- 
cated by the solid line in FIG. 1 1 . Inside the high-resolu- 
tion picture updating unH 69. the second upsampling 
unit 69b applies an upsampling process to convert the 
reconstructed prediction error signal supplied from the 
IDCT processor 67b. thus producing a high-resolution 
(CIF) version of the reconstructed prediction error sig- 
nal. Subsequently, the adder 69c adds the upsampled 
reconstructed prediction error signal to a corresponding 
high-resolution (CIF) block image retrieved from the 
high-resolution picture storage unit 62 so as to obtain a 
high-resolution (CIF) reconstructed block image. This 
block image is then fed back to the high-resolution pic- 
ture storage unit 62 via the selection switch 69d and the 
switch 71. A corresponding block image stored in the 
high-resolution picture storage unit 62 is now replaced 
with that new block image having the high resolution 
(CIF). Such updating process permits the high-resolu- 
tion picture storage unit 62 to keep the latest picture 
even while the main decoder loop is working in the low 
resolution (OCIF) mode. Because the above-described 
block updating operations are based on a high-resolu- 
tion (CIF) picture, the updated blocks can keep their 
original quality levels, as well as permitting temporal 
changes in luminance to be properly reflected. 

Consequently, the background portions of video 
images in the high-resolution picture storage unit 62 are 
preserved at high quality levels, regardless of the reso- 



lution modes. The video decoder of FIG. 10 uses this 
high-resolution picture storage unit 62 as the signal 
souice for the video display unit 70. Although the pictur© 
resolution may be reduced at the sending end to regu- 
5 late the amount of video transmission data within a pre- 
determined standard level, the video display unit 70 
provides clear video images whose inactive background 
portions are maintained at high quality levels, without 
being affected by reduction of the picture resolution. 
io Besides succeeding to the bask: structure of a con- 
ventional decoder loop, the video decoder according to 
the third embodiment accomplishes its effect by simply 
introducing a few additional circuits, such as the high- 
resolution picture storage unit 62, high-resolution p5c- 
15 ture updating unit 69. switch 71, and selection switch 



The above discussion is now summarized below. 
As important features of the present invention, the pro- 
posed video coder comprises a high-resolution pictur® 
20 storage unit, low resolution pictur® storage uret, high- 
resolution picture updating unit, and tew-r®solution pic- 
ture updating unit These units maintain tho quality of 
inactive regions of a pictur©. such as background 
images, preventing them from being degraded when th® 
25 resolution mode has changed from h5gh to tea. 

In conventional coders, a low-to-high transition in 
the resolution mode invokes retransmission of detailed 
visual information of background imag<s@. Th® victeo 
coder proposed in the present invention, however, ©lim- 
30 inates the need for sending such details, thus suppress- 
ing unwanted increase in size of coded data transmitted 
over the communications channel. 

The proposed video coder classifies coded blocks 
into active coded blocks and static coded blocks, and 
35 maintains the resolution and quality of the static coded 
blocks. This enables appropriate processes to b® 
applied to such coded blocks that have some luminance 
change but no spatial motion. 

Furthermore, the proposed video decoder prevents 
40 degradation in picture qualities from happening to inac- 
tive regions of a picture, such as background images, 
when the resolution mode has changed from high to 
low. The present invention accomplishes this by simpJy 
introducing a high-resolution picture storage unit and a 
45 high-resolution picture updating unit, besides exploiting 
a conventional decoder loop configuration. 

The foregoing is considered as illustrative only of 
the principles of the present invention. Further, since 
numerous modifications and changes will readily occur 
so to those skilled in the art, it is not desired to limit th® 
invention to the exact construction and applications 
shown and described, and accordingly, all suitable mod- 
ifications and equivalents may be regarded as falling 
within the scope of the invention in the appended claims 
55 and their equivalents. 
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Claims 

1. A video coding apparatus for performing a predic- 
tive coding of digital video input signals in conjunc- 
tion with an internal picture format conversion 
according to a picture resolution mode, which is 
either a high resolution mode or a low resolution 
mode determined by a resolution selection control- 
ler disposed as an integral part of the video coding 
apparatus, the video coding apparatus comprising: 

(a) high-resolution picture storage means for 
storing a high-resolution picture that has been 
locally reconstructed in a high resolution for- 
mat; 

(b) low-resolution picture storage means for 
storing a low-resolution picture that has been 
locally reconstructed in a low resolution format; 

(c) selective reading-out means for selectively 
reading out the high-resolution picture from 
said high-resolution picture storage means 
when the high resolution mode has been 
selected by the resolution selection controller, 
or the low-resolution picture from said low- res- 
olution ptcture storage means when the low 
resolution mode has been selected by the res- 
olution selection controller; 

(d) high-resolution picture updating means for 
converting the low-resolution picture retrieved 
from said low-resolution picture storing means 
into a high-resolution image when the low res- 
olution mode is effective, and storing the result- 
ant high-resolution image into said high- 
resolution picture storage means; and 

(e) low-resolution picture updating means for 
converting the high-resolution picture retrieved 
from said high-resolution picture storage 
means into a low-resolution picture when the 
resolution selection controller has changed the 
picture resolution mode from the high resolu- 
tion mode to the low resolution mode, and stor- 
ing the resultant low-resolution picture into said 
low-resolution picture storage means. 

The video coding apparatus according to claim 1, 
wherein 

the video coding apparatus has transform, 
quantization, dequantization, inverse transform, 
and motion compensation functions, and 

said high-resolution picture updating means 
comprises: 
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(dl) first updating means for converting the 
low-resolution picture retrieved from said low- 
resolution picture storage means to obtain 55 
high-resolution block images corresponding to 
coded blocks with non-zero motion vectors and 
intra-coded blocks while the low resolution 



mode is effective, and storing the high-resolu- 
tion block images to said high-resolution pic- 
ture storage means, and 
(62) second updating means for converting, 
while the low resolution mode is effective, a 
prediction error signal having been reproduced 
by using the dequantization and inverse trans- 
form functions, to obtain a high-resolution pre- 
diction error signal corresponding to coded 
blocks with zero motion vectors, reconstructing 
high-resolution block images by adding the 
high-resolution prediction error signal to corre- 
sponding blocks of the high-resolution picture 
retrieved from the high-resolution picture stor- 
age means, and updating the high-resolution 
picture stored in the high- resolution picture 
storage means with the reconstructed high-res- 
olution block images. 



20 3. A 



video decoding apparatus for receiving and 
decoding a predictive-coded video brtstream pro- 
duced by compressing digital video input signals 
_^ itn ^ predictive coding technique in conjunction 
witffan ihternarpicture format conversion according 
to a picture resolution mode determined by a reso- 
lution selection controller, the video decoding appa- 
ratus comprising: 

(a) high-resolution picture storage means for 
storing a high-resolution picture that has been 
reconstructed in a high resolution format; 

(b) low-resolution picture storage means for 
storing a low-resolution picture that has been 
reconstructed in a low resolution format; 

(c) selective reading-out means, responsive to 
the picture resolution mode that has been 
decoded, for selectively reading out the high- 
resolution picture from said high-resolution pic- 
ture storage means when a high resolution 
mode is effective, or the low-resolution picture 
from said low-resolution picture storage means 
when a low resolution mode is effective; 

(d) high-resolution picture updating means for, 
when the decoded picture resolution mode 
indicates that the low resolution mode is effec- 
tive, converting the low-resolution picture 
retrieved from said low-resolution picture stor- 
age means to obtain high-resolution block 
images corresponding to coded blocks, and 
storing the high-resolution block images into 
said high-resolution picture storage means; 
and 

(e) low-resolution picture updating means for 
converting the high-resolution picture retrieved 
from said high-resolution picture storage 
means to obtain a low-resolution picture when 
the decoded picture resolution mode has 
changed from the high resolution mode to the 
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low resolution mode, and storing the obtained 
low-resolution picture into said low-resolution 
picture storage means. 

The video decoding apparatus according to claim 3. 5 
wherein 

the video coding apparatus has dequantiza- 
tion and inverse transform functions, and 

said high-resolution picture updating means 
comprises: 10 

(dl) first updating means tor converting the 
low-resolution picture retrieved from said low- 
resolution picture storage means to obtain 
high-resolution block images corresponding to 15 
coded blocks with non-zero motion vectors and 
intra-coded blocks while the low resolution 
mode is effective, and storing the high-resolu- 
tion block images to the high-resolution picture 
storage means, and 20 
(d2) second updating means for converting, 
while the low resolution mode is effective, a 
prediction error signal having been reproduced 
by the dequantization and inverse transform 
functions, to obtain a high-resolution prediction 25 
error signal corresponding to coded blocks with 
zero motion vectors, reconstructing high-reso- 
lution block images by adding the high-resolu- 
tion prediction error signal to corresponding 
blocks of the high-resolution picture retrieved so 
from the high-resolution picture storage 
means, and updating the high-resolution pic- 
ture stored in the high-resolution picture stor- 
age means with the reconstructed high- 
resolution block images. 35 



5. 



A video decoding apparatus for receiving and 
decoding a predictive-coded video bitstream pro- 
duced by compressing digital video input signals 
with a predictive coding technique in conjunction 
with an internal picture format conversion according 
to a picture resolution mode determined by a reso- 
lution selection controller, the video decoding appa- 
ratus comprising: 

(a) high-resolution picture storage means for 
storing a high-resolution picture that has been 
reconstructed in a high resolution format; 

(b) decoded picture storage means for storing 
a decoded picture that has been reconstructed; 

(c) decoded picture generating means for pro- 
ducing the decoded picture based on a repro- 
duced prediction error signal, storing the 
decoded picture into said decoded picture stor- 
age means, and additionally storing the 
decoded picture into said high-resolution pic- 
ture storage means only when the decoded 
picture is of the high resolution format; and 
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(d) high-resolution picture updating means for, 
when the decoded picture is of a low resolution 
format, selectively converting coded blocks of 
the decoded picture to obtain high-resolution 
block images of the coded blocks, and storinQ 
the obtained high-resolution block images into 
said high-resolution picture storage means. 

>. The video decoding apparatus according to claim 5, 
wherein 

the video coding apparatus has dequantiza- 
tion and inverse transform functions, and 

said high-resolution picture updating means 
comprises: 

(dl) first updating means tor converting the 
decoded picture supplied from said decoded 
pictures generating means to obtain high-reso- 
lution block images corresponding to coded 
blocks with non-zero motion vectors and intra- 
coded blocks when the decoded picture is of 
the low resolution format and storing the high- 
resolution block images to the high-resolution 
picture storage means, and 
(d2j second "updating" means tor. when the 
decoded picture is of the low resolution format, 
converting a prediction error signal having 
been reproduced by the dequantization and 
inverse transform functions, to obtain a high- 
resolution prediction error signal corresponding 
to coded blocks with zero motion vectors, 
reconstructing high-resolution block images by 
adding the high- resolution prediction error sig- 
nal to corresponding blocks of the high-resolu- 
tion picture retrieved from said high-resolution 
picture storage means, and updating the high- 
resolution picture stored in said high-resolution 
picture storage means with the reconstructed 
high-resolution block images. 

7. A video coding method for predictively coding a 
video input signal, which is implemented in a video 
coding apparatus having a high-resolution picture 
storage unit and a low resolution picture storage 
unit, the video coding method comprising the steps 
of: 

(51) storing a decoded picture that has been 
locally reconstructed in a previous frame into 
the high-resolution picture storage unit H the 
decoded picture is a high-resolution picture, or 
storing the decoded picture to the low-resolu- 
tion picture storage unit H the decoded picture 
is a low-resolution picture; 

(52) determining which picture resolution 
mode, a high resolution mode or a low resolu- 
tion mode, is to be used to encode the video 
input signal in a present frame; 
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(53) reading out the high-resolution picture 
from the high-resolution picture storage unit 
when the high resolution mode has been 
selected in said step (S2), or reading out the 
low-resolution picture from the low-resolution 5 
picture storage unit when the low resolution 
mode has been selected in said step (S2); 

(54) obtaining a decoded picture of the present 
frame by adding a prediction error signal to a 
prediction picture constructed from the high- io 
resolution picture or the low- resolution picture 
read out in said step (S3), and storing the 
obtained decoded picture into the high-resolu- 
tion picture storage unit when the high resolu- 
tion mode has been selected in the step (S2), is 
or into the low-resolution picture storage unit 
when the low resolution mode has been 
selected in the step (S2); 

(55) converting the low-resolution picture 
retrieved from the low-resolution picture stor- 20 
age unit to obtain high-resolution block images 
corresponding to coded blocks when the low 
resolution mode has been selected in the step 
(S2). and storing the high -resolution block 
images to the high-resolution picture storage 25 
unit; 

(56) converting the high-resolution picture 
retrieved from the high-resolution picture stor- 
age unit into a low-resolution picture when the 
picture resolution mode determined in the step so 
(S2) has changed from the high resolution 
mode to the low resolution mode, and storing 
the low-resolution picture into the low-resolu- 
tion picture storage unit 

35 

8. The video coding method according to claim 7, 
wherein said step (S5) comprises the substeps of: 

(S5a) when the low resolution mode has been 
selected in the step (S2) t converting the low- 40 
resolution picture retrieved from the low-resolu- 
tion picture storage unit to obtain high-resolu- 
tion block images corresponding to coded 
blocks with non-zero motion vectors and intra- 
coded blocks, and storing the high-resolution 45 
block images to the high-resolution picture 
storage unit, and 

(S5b) when that the low resolution mode has 
been selected in the step (S2), converting a 
prediction error signal having been reproduced so 
by using dequantization and inverse transform 
functions, to obtain a high-resolution prediction 
error signal corresponding to coded blocks with 
zero motion vectors, reconstructing high-reso- 
lution block images by adding the high-resolu- 55 
tion prediction error signal to corresponding 
blocks of the high-resolution picture retrieved 
from the high-resolution picture storage unit 



and updating the high-resolution picture stored 
in the high-resolution picture storage unit with 
the reconstructed high-resolution block 
images. 

I. A video decoding method for receiving and decod- 
ing a predictive-coded video bitstream. which is 
implemented in a video decoding apparatus having 
a high-resolution picture storage unit and a low res- 
olution picture storage unit, the video decoding 
method comprising the steps of: 

(51) storing a decoded picture that has been 
reconstructed in a previous frame into the high- 
resolution picture storage unit if the decoded 
picture is a high-resolution picture, or storing 
the decoded picture to the low- resolution pic- 
ture storage unit if the decoded picture is a low- 
resolution picture; 

(52) decoding a picture resolution mode that 
indicates which of a low resolution mode and a 
high resolution mode is effective for decoding 
the present frame, and selectively reading out 
the high-resolution picture from the high-reso- 
lution picture storage unit when the high resolu- 
tion mode is effective, or the low-resolution 
picture from the low-resolution picture storage 
unit when the low resolution mode is effective; 

(53) converting the low-resolution picture 
retrieved from the low-resolution picture stor- 
age unit to obtain high-resolution block images 
corresponding to coded blocks when the low 
resolution mode is effective, and storing the 
high-resolution block images to the high-reso- 
lution picture storage unit; and 

(54) if the picture resolution mode has changed 
from the high resolution mode to the low reso- 
lution mode, converting the high-resolution pic- 
ture retrieved from the high-resolution picture 
storage unit into a low-resolution image and 
storing the low-resolution Image into the low- 
resolution picture storage unit. 

10. The video decoding method according to claim 9, 
wherein said step (S3) comprises the substeps of: 

(S3a) when the low resolution mode is effec- 
tive, converting the low-resolution picture 
retrieved from the low-resolution picture stor- 
age unit to obtain high-resolution block images 
corresponding to coded blocks with non-zero 
motion vectors and intra-coded blocks, and 
storing the high-resolution block images to the 
high-resolution picture storage unit and 
(S3b) when the low resolution mode is effec- 
tive, converting a prediction error signal having 
been reproduced by using dequantization and 
inverse transform functions, to obtain a high- 



16 



D: <£P 088329SA2J_> 



31 EP 0 883 299 A2 32 

resolution prediction error signal corresponding picture storage unit with the reconstructed 

to coded blocks with zero motion vectors. high-resolution block images, 

reconstructing high- resolution block images by 
adding the high-resolution prediction error sig- 
nal to corresponding blocks of the high-resolu- s 
tion picture retrieved Irom the high-resolution 
picture storage unit, and updating the high-r es- 
olution picture stored in the high-resolution pic- 
ture storage unit with the reconstructed high- 
resolution block images. io 

11. A video decoding method for receiving and decod- 
ing a predictive-coded video signal, which is imple- 
mented in a video decoding apparatus having a 
decoded picture storage unit and a high-resolution is 
picture storage unit, the video decoding method 
comprising the steps of: 

(51) storing a decoded picture that has been 
reconstructed in a previous frame into the high- 20 
resolution picture storage unit; 

(52) reconstructing a decoded picture of the 
present frame from a reproduced prediction 
error signal, and storing the decoded picture 
into the decoded picture storage unit, and addi- 25 
tionally storing the same decoded picture into 
the high-resolution picture storage unit only 
when the decoded picture has a high resolu- 
tion; and 

(53) converting the decoded picture to obtain a so 
high-resolution block images corresponding to 
coded blocks when the decoded picture has a 
low resolution, and storing the high-resolution 
block images to the high-resolution picture 
storage unit 35 

12. The video decoding method according to claim 11, 
wherein said step (S3) comprises the substeps of: 

(S3a) when the decoded picture has the low 40 
resolution, converting the decoded picture to 
obtain high-resolution block images corre- 
sponding to coded blocks with non-zero motion 
vectors and intra-coded blocks, and storing the 
high-resolution block images to the high-reso- 45 
lution picture storage unit, and 
(S3b) when the decoded picture has the low 
resolution, converting a prediction error signal 
having been reproduced by using dequantiza- 
tion and inverse transform functions, to obtain a so 
high-resolution prediction error signal corre- 
sponding to coded blocks with zero motion vec- 
tors, reconstructing high-resolution block 
images by adding the high-resolution predic- 
tion error signal to corresponding blocks of the ss 
decoded picture retrieved from the high-resolu- 
tion picture storage unit, and updating the 
decoded picture stored in the high-resolution 
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